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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
Appellants: Keren et al. Attorney Docket No: 150824.04 

Serial No: 09/770,767 Group Art Unit: 2621 

Filed: January 25, 2001 Examiner: Czekaj, DavidJ. 

Title: MULTIMEDIA STREAM COMPRESSION 

APPEAL BRIEF UNDER 37 C.F.R. $ 41.37 



MAIL STOP APPEAL BRIEF - PATENTS 
P.O. Box 1450 
Alexandria, VA 22313-1450 

Sir: 



This is an appeal from an Office Action dated August 11, 2006, twice rejecting Claims 94- 
111 and 128-139. Appellants filed a timely Notice of Appeal on January 11, 2007. 

The jurisdiction of this board is invoked under the provisions of 35 U.S.C. § 134 and 37 
C.F.R. Part 41. 
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1. REAL PARTY IN INTEREST 

The real party in interest in this appeal is hereby identified as Microsoft Corporation, since 
all right and title in the invention and the patent application on appeal are owned by Microsoft 
Corporation, as evidenced by a chain of title from the inventors in the patent application identified 
above to the current assignee, as shown below. 

1. An assignment of all rights and title in the present patent apphcation by inventors 
Avishai Keren, Meir Feder, and Ofir Paz to WebTV Networks, Inc. The assignment was 
recorded in the U.S. Patent and Trademark Office on April 4, 2001 at Reel 01 1792, Frame 0934. 

2. An assignment of all rights based on a merger WebTV Networks, Inc. to Microsoft 
Corporation. The assignment was recorded in the U.S. Patent and Trademark Office on May 2, 
2006, at Reel 017561, Frame 0603. 
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2. RELATED APPEALS AND INTERFERENCES 

No other appeals or interferences are known to appellants that will directly affect or be 
directly affected by or have a bearing on the Board's decision in this pending appeal. 
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3. STATUS OF THE CLAIMS 

Claims 94-1 1 1 and 128-139 are remain pending in the application on appeal, claims 1-93 and 
112-127 having been previously canceled. No claims have been allowed. Claims 94-111 and 128- 
1 3 9 have been rej ected under 35U.S.C.§103(a). 

In the final office action dated August 11, 2006, claims 94-99 and 137 were rejected under 35 
U.S.C. § 103(a) as being unpatentable over U.S. Patent No. 6,278,466 ("Chen") in view of U.S. 
Patent No. 6,195,692 ("Hsu"). Claims 100-101 were rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Chen and Hsu in further view of U.S. Patent No. 5,623,587 ("Bulman"). Claims 
102-1 1 1 were rejected under 35 U.S.C. § 103(a) as being unpatentable over Chen, Hsu, and Bulman 
in further view of U.S. Patent No. 6,175,663 ("Huang"). Claims 128-136 and 138-139 were rejected 
under 35 U.S.C. § 103(a) as being unpatentable over Chen and Hsu in further view of U.S. Patent 
No. 6,445,874 ("Catlow"). 
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4. STATUS OF THE AMENDMENTS 

An Amendment and Reply dated May 16, 2006, was filed by the applicant after an interview 
with the examiner. A final Office Action was mailed on August 11, 2006. No fiirther amendment 
has been filed. 

A copy of the claims on appeal, including all amendments actually entered, is appended 

hereto. 
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5. SUMMARY OF CLAIMED SUBJECT MATTER 

Independent Claim 94 

Independent claim 94 recites a method in a server 16 (shown in Figs. 1 and 2) that provides 
remote client access (for example, remote client 1 1 in Fig. 1) to one or more computer programs 30 
(shown in Fig. 2) in order to allow remote user interaction therewith. The method provides for 
generating a compressed video stream representing a display for the one or more computer programs 
by modifying elements of the display for efficient compression. See, for examples, the arrows 
labeled "compressed video" and "user control" in Fig. 1 . 

The method provides for generating display elements at a server, the display elements 
representing at least a portion of a display for a computer program running at the server, which is 
configured to receive remote interaction from a remote client. See the server 16 including programs 
30 which generate display elements for a display and which is configured to receive interaction from 
a client 1 1 . See Figs. 1 and 2 and its description in paragraph 147. 

The method requires receiving, at the computer program, continuous user interaction via the 
remote client that changes one or more elements of the display for said executed computer program, 
wherein at least one of the display elements changed is an unmodified object. Examples of 
continuous user interaction is provided throughout the specification. For example, paragraph 170 
describes the scrolling modification performed responsive to continuous user interaction. See also 
paragraph 173 which describes how the changes caused by user interaction are detected. Paragraph 
191 also discloses that the method described with respect to Fig. 3 is performed responsive to change 
events which indicate to the image compressor that a change has occurred in an image of a display 
channel (i.e., a displayed image). 

The method requires modifying at least said unmodified object to produce a modified object, 
wherein said modification is performed independently of said executed computer program in such a 
way that the modified object will be more efficiently converted to a compressed video stream than 
the unmodified object. Steps 50, 56, 58, 60, and 62 in Fig. 3 describe how each display element is 
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modified so that modified object is more efficiently converted to a compressed video stream. Details 
of these method steps are provided, for example, in paragraphs 160, 163, 164, 166-173, and 181-183. 

The method also requires generating a compressed video stream by converting at least the 
modified object into the compressed video stream for rendering at least a portion of said display of 
the executed computer program at a display device at the client for allowing a user interaction with 
the at least one of the display elements. Steps 66-70 in Fig. 3 and its description, for example, in 
paragraphs 181-187 describe how the compressed video stream is generated for rendering a portion 
of a display of a computer program 30 by converting a modified object into the compressed video 
stream. 

Independent Claim 128 

Independent claim 128 recites that in a server (server 16 in Figs. 1 and 2) that provides 
remote client (chent 1 1 in Fig. 1) access to one or more computer programs 30 (in Fig. 2), a method 
is provided of determining when to generate a compressed video stream representing a display for 
the one or more computer programs by monitoring changes to the display. See, for examples, the 
arrows labeled "compressed video" and "user control" in Fig. 1 . 

The method includes executing a computer program 30 at a server 16 (see Fig. 2), wherein 
the execution of the computer program generates display objects from a set of display commands, 
the display objects are at least a portion of a display for said computer program. See Figs. 1 and 2 
and its description in paragraph 147. 

The method includes identifying changes of said display which are responsive to at least one 
type of continuous user interaction command received from a remote client. Examples of 
continuous user interaction is provided throughout the specification. For example, a paragraph 170 
describes the scrolling modification performed responsive to continuous user interaction. See also 
paragraph 173 which describes how the changes caused by user interaction are detected. Paragraph 
191 also discloses that the method described with respect to Fig. 3 is performed responsive to change 
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events which indicate to the image compressor that a change has occurred in an image of a display 
channel (i.e., a displayed image). 

The method includes determining whether the changes warrant an update to an image based 
at least in part on one or more of available bandwidth, available computing power, or type of user 
connection. See paragraph 191 which describes change detection based on available bandwidth, 
available computing power, and/or type of user connection which warrants an update to an image. 

The method also requires that upon determining that the changes do warrant an update, 
processing the changes and converting said display commands into a compressed video stream, 
wherein said changes are inserted into said compressed video stream at an update frame rate 
corresponding to a priority assigned to other portions of the display that are unchanged and such that 
changes to said image are inserted into the compressed video stream at a faster rate than compressed 
data that does not include changes to said image. This feature is described generally with respect to 
the steps of Fig. 3 and especially in paragraphs 200-202 which describes how a higher priority may 
be accorded to change events related to user interaction in which modified objects corresponding to 
these change events may be inserted at a higher frame rate. 
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6. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

I. A determination as to whether to whether the examiner erred in rejecting: (a) claim 94-99 
and 137 under 35 U.S.C. § 103(a) as being unpatentable over U.S. Patent No. 6,278,466 ("Chen") in 
view of U.S. Patent No. 6,195,692 ("Hsu"); (b) claims 100-101 under 35 U.S.C. § 103(a) as being 
unpatentable over Chen and Hsu in further view of U.S. Patent No. 5,623,587 ("Bulman"); (c) 
claims 102-111 under 35 U.S.C. § 103(a) as being unpatentable over Chen, Hsu, and Bulman in 
further view of U.S. Patent No. 6,175,663 ("Huang"); and (d) claims 132, 134-136, and 138 over 
Chen and Hsu in further view of U.S. Patent No. 6,445,874 ("Catlow"). 

II. A determination as to whether the examiner erred in rejecting Claims 128-131, 133, and 
139 under 35 U.S.C. § 103(a) as being unpatentable over Chen and Hsu in further view of Catlow. 
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7. ARGUMENT 

I. THE EXAMINER ERRED IN REJECTING INDEPENDENT CLAIM 94 UNDER 



35 U.S.C. § 103(a) BECAUSE THE CITED ART FAILS TO TEACH ALL OF THE CLAIMED 
FEATURES 

CLAIM 94 IS PATENTABLE UNDER 35 U.S.C. § 103(a), BECAUSE THE CITED ART 
FAILS TO TEACH (a) CONTINUOUS USER INTERACTION WITH A COMPUTER 
PROGRAM AT A SERVER THAT GENERATES DISPLAY ELEMENTS WHERE (b) AN 
UNMODIFIED OBJECT IS MODIFIED AT THAT SERVER INDEPENDENT OF THE 
EXECUTED COMPUTER PROGRAM 

Independent claim 94 recites, in a server , 

generating display elements at a server, the display elements representing at least a 
portion of a display for a computer program running at the server, which is configured to 
receive remote interaction from a remote client; 

receiving, at the computer program, continuous user interaction via the remote client 
that changes one or more elements of the display for said executed computer program, 
wherein at least one of the display elements changed is an unmodified object; 

modifying at least said unmodified object to produce a modified object, wherein said 
modification is performed independently of said executed computer program in such a way 
that the modified object will be more efficiently converted to a compressed video stream than 
the unmodified object; and 

generating a compressed video stream by converting at least said modified object into 
said compressed video stream for rendering at least a portion of said display of the executed 
computer program at a display device at the client for allowing a user interaction with the at 
least one of the display elements. 

Therefore, claim 94 requires that display elements are generated by a computer 
program at a server where the computer program receives continuous user interaction from a 
remote client that changes one or more elements of the display of the executed computer 
program where one of the display elements changed is an unmodified object. The claim 
further requires, at that server , modifying an unmodified object to produce a modified object 
independently of the executed computer program. This combination of features is not taught 
by the applied prior art. 

First, as acknowledged in the final rejection, Chen fails to disclose the continuous 
user interaction as claimed. 

Second, Chen also does not disclose modifying an unmodified object at the server 
where the display elements are generated. Specifically, the office action states that Chen 
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discloses modifying an object to more efficiently convert to a compressed stream in col. 5, 
lines 36-58 and col. 19, lines 10-35. 

However, col. 5, lines 36-58 discloses that any such modifications is performed by 
the server system 16 which is very different from the animation authoring system 12 which 
generates the display elements (elements of animation generated by the system of Chen) . 
Likewise, in col. 19, lines 10-35, Chen discloses how the server system 16 controls aspects of 
how the animation data stream is delivered to the playback systems 18A-C. Nowhere does 
Chen teach or suggest that the modifying of an unmodified object be performed at the server 
where the current program (which generates the display objects) executes as required by 
independent claim 94. 

To cure these deficiencies in Chen, the office action relies on Hsu. However, Hsu 
does not disclose anything with respect with respect to the second deficiency in Chen 
discussed above. Specifically, Hsu relates to presenting a user a set of links which is used to 
provide the user the internet content (such as a webpage) that is referenced by that link. See, 
for example, col. 8, lines 27-35 of Hsu. While Hsu discloses user interaction for an internet 
on-demand system on a television, it does not disclose any of the other features recited in 
independent claim 94. 

Therefore, the alleged prior art of Chen, Hsu, Bulman, Huang, and Catlow — taken 
either individually or as a whole — does not disclose, suggest, or enable (among other things) 
generating display elements at a server, the display elements representing at least a portion of 
a display for a computer program running at the server, which is configured to receive remote 
interaction from a remote client, receiving, at the computer program, continuous user 
interaction via the remote client that changes one or more elements of the display for the 
executed computer program, wherein at least one of the display elements changed is an 
unmodified object, and modifying the unmodified object to produce a modified object, 
wherein the modification is performed independently of the executed computer program in 
such a way that the modified object will be more efficiently converted to a compressed video 
stream than the unmodified object. 

Since several of the recited features in independent claim 94 are not disclosed or 
suggested by any of the applied prior art, the office action fails to make a prima facie case of 
obviousness with respect to the pending independent claim 94. 

■ ^ ^ ■ Application Serial No: 091110,161 
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In this context it should be noted that The Patent Office (PTO) has the burden of proving 
each of the claimed features is shown by the prior art. An allegation that claimed subject matter is 
"obvious" (as here alleged) requires a positive, concrete teaching in the prior art, such as would lead 
a person skilled in the art to choose the claimed combination from among many that might be 
comprehended by broad prior art teachings. The PTO's review court has made it very clear that 
silence in a reference is hardly a substitute for clear and concrete evidence from which a conclusion 
of obviousness might justifiably flow. See, e.g.. Application of Burt, 356 F.2d 115, 121 (CCPA 
1966). 

DEPENDENT CLAIMS 95-111, 132, and 134-138 ARE PATENTABLE FOR AT 
LEAST THE SAME REASONS AS THE INDEPENDENT CLAIM 94 ON WHICH THEY 
DIRECTLY OR INDIRECTLY DEPEND 

In addition, these claims recite additional features which are also patentable when 
considered as a whole. 

II. THE EXAMINER ERRED IN REJECTING CLAIMS 128-131. 133. AND 139 
UNDER 35 U.S.C. § 103(A) AS BEING UNPATENTABLE OVER CHEN AND HSU IN 
FURTHER VIEW OF CATLOW. 

Independent claim 128 recites, in a server : 

executing a computer program at a server, wherein the execution of the computer 
program generates display objects from a set of display commands , the display objects are at 
least a portion of a display for said computer program; 

identifying changes of said display which are responsive to at least one type of 
continuous user interaction command received from a remote client 

determining whether said changes warrant an update to an image based at least in 
part on one or more of available bandwidth, available computing power, or type of user 
connection : and 

upon determining said changes do warrant an update, processing said changes and 
converting said display commands into a compressed video stream, wherein said changes are 
inserted into said compressed video stream at an update frame rate corresponding to a priority 
assigned to other portions of the display that are unchanged and such that changes to said 
image are inserted into the compressed video stream at a faster rate than compressed data that 
does not include changes to said image . 

As discussed earlier herein with respect to claim 94, several of the features recited in claim 
128 are not disclosed or suggested by the applied combination of Chen and Hsu. Speciflcally, 
nowhere does the combination of Chen or Hsu disclose or suggest (a) in a server , executing a 
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computer program at a server, wherein the execution of the computer program generates display 
objects from a set of display commands, where the display objects are at least a portion of a display 
for said computer program, and (b) in that server, determining whether said changes warrant an 
update to an image based at least in part on one or more of available bandwidth, available computing 
power, or type of user connection. 

This combination of features is not taught by the applied prior art. 

First, as acknowledged in the final rejection, Chen fails to disclose the continuous 
user interaction as claimed. 

Second, Chen also does not determining whether said changes (responsive to a user 
interaction) warrant an update to an image based at least in part on one or more of available 
bandwidth, available computing power, or type of user connection. Specifically, the office 
action states that Chen discloses modifying an object to more efficiently convert to a 
compressed stream in col. 5, lines 36-58 and col. 19, lines 10-35. 

However, col. 5, lines 36-58 discloses that any such modifications is performed by 
the server system 16 which is very different from the animation authoring system 12 which 
generates the display elements (elements of animation generated by the system of Chen) . 
Likewise, in col. 19, lines 10-35, Chen discloses how the server system 16 controls aspects of 
how the animation data stream is delivered to the playback systems 18A-C. Nowhere does 
Chen teach or suggest that the modifying of an unmodified object be performed at the server 
where the current program (which generates the display objects) executes as required by 
independent claim 94. 

To cure these deficiencies in Chen, the office action relies on Hsu. However, Hsu 
does not disclose anything with respect with respect to the second deficiency in Chen 
discussed above. Specifically, Hsu relates to presenting a user a set of links which is used to 
provide the user the internet content (such as a webpage) that is referenced by that link. See, 
for example, col. 8, lines 27-35 of Hsu. While Hsu discloses user interaction for an internet 
on-demand system on a television, it does not disclose any of the other features recited in 
independent claim 128. 

Third, the independent claim 128 recites wherein said changes to display commands 
are inserted into said compressed video stream at an update frame rate corresponding to a 
priority assigned to other portions of the display that are unchanged and such that changes to 
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said image are inserted into the compressed video stream at a faster rate than compressed 
data that does not include changes to said image . The office action acknowledges that 
neither Chen nor Hsu disclose this feature but relies on Catlow. 

However, Catlow discusses priority in the context of priority between clips which are 
defined as a continuous sequence of video frames. See col. 1, lines 14-16. Nowhere does 
Catlow disclose anything related to the claimed automated changing of priority to a changed 
display command so that an update frame of the changes is inserted at a frame rate that is 
faster than compressed data that does not include such changes. Accordingly, this feature is 
also not disclosed by the apphed prior art and provides an additional reason for the 
patentability of independent claim 128. 

Accordingly, since several of the claimed features are not disclosed or suggested by 
the applied prior art, the office action fails to make a prima facie case of obviousness with 
respect to independent claim 128. 

DEPENDENT CLAIMS 129-131, 133, AND 139 ARE PATENTABLE FOR AT LEAST 
THE SAME REASONS AS THE INDEPENDENT CLAIM 128 ON WHICH THEY 
DIRECTLY OR INDIRECTLY DEPEND 

In addition, these claims recite additional features which are also patentable when 
considered as a whole. 
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CONCLUSION 



Appellants therefore respectfully request that the Board of Patent Appeals and Interferences 
overrule the Examiner's rejection of the claims and require that the Examiner pass this case to issue 
without further delay. 

Respectfully submitted, 



Dated: April 9. 2007 By: /Aaron.C. Chatteriee/ 

Aaron C. Chatterjee 

Microsoft Corp. 
Reg. No. 41,398 
(703) 647-6572 
One Microsoft Way, 
Redmond, WA 98052 



CERTIFICATE OF MAILING OR TRANSMISSION [37 CFR 1.8(a)1 

I hereby certify that this correspondence and the documents identified on this form are being 
electronically deposited with the USPTO via EFS-Web on the date shown below: 



April 9. 2007 /Kate Marochkina/ 

)ate Kate Marochkina 



-15- 



Application Serial No: 09/770,767 
Attorney Docket No: 150824.04 
Microsoft Corp. 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 



8. CLAIMS APPENDIX 

A copy of the claims on appeal 



94. In a server that provides remote client access to one or more computer programs in order to 
allow remote user interaction therewith, a method of generating a compressed video stream 
representing a display for the one or more computer programs by modifying elements of the display 
for efficient compression, the method comprising: 

generating display elements at a server, the display elements representing at least a portion of 
a display for a computer program running at the server, which is configured to receive remote 
interaction from a remote client; 

receiving, at the computer program, continuous user interaction via the remote chent that 
changes one or more elements of the display for said executed computer program, wherein at least 
one of the display elements changed is an unmodified object; 

modifying at least said unmodified object to produce a modified object, wherein said 
modification is performed independently of said executed computer program in such a way that the 
modified object will be more efficiently converted to a compressed video stream than the 
unmodified object; and 

generating a compressed video stream by converting at least said modified object into said 
compressed video stream for rendering at least a portion of said display of the executed computer 
program at a display device at the chent for allowing a user interaction with the at least one of the 
display elements. 

95. A method according to claim 94, wherein said modifying comprises modifying said 
unmodified object responsive to limitations of said display device on which said compressed video 
stream is to be displayed. 
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96. A method according to claim 94, wherein modifying comprises modifying said unmodified 
object to reduce bandwidth requirements of said compressed video stream. 

97. A method according to claim 94, wherein modifying comprises modifying said unmodified 
object to reduce resource requirements for compressing said compressed video stream. 

98. A method according to claim 94, wherein said modifying comprises changing said 
unmodified object such that the unmodified object is moved relative to its original display position. 

99. A method according to claim 98, wherein said compressed video stream utilizes blocks 
having boundaries and wherein moving comprises moving said unmodified object to match at least 
one compression block boundary. 

100. A method according to claim 94, wherein said modifying comprises replacing said 
unmodified object with a different object to produce said modified object. 

101 . A method according to claim 100, wherein replacing said unmodified object comprises 
replacing said unmodified object with a compressed representation of said different object. 

102. A method according to claim 100, wherein said unmodified object comprises a text object. 

103. A method according to claim 100, wherein said unmodified object comprises a background 
of said display. 

104. A method according to claim 100, comprising analyzing said unmodified object to determine 
a closest suitable replacement object for producing said modified object. 
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105. A method according to claim 94, wherein modifying said unmodified object comprises 
changing a font definition for said unmodified object. 

106. A method according to claim 94, wherein modifying said unmodified object comprises 
modifying at least one color of said unmodified object. 

107. A method according to claim 106, wherein modifying a color composes reducing a spatial 
resolution of said colors. 

108. A method according to claim 106, wherein modifying a color composes reducing a color 
range resolution of said colors. 

109. A method according to claim 94, wherein modifying said unmodified object comprises 
reducing a spatial resolution of said unmodified object. 

110. A method according to claim 94, wherein said unmodified object comprises a scrolling 
command and wherein said modifying comprises increasing a granularity of said scrolling. 

111. A method according to claim 110, wherein said increasing a granularity comprises limiting 
said scrolling command to multiples of compression blocks size of said compressed stream. 

128. In a server that provides remote client access to one or more computer programs, a method of 
determining when to generate a compressed video stream representing a display for the one or more 
computer programs by monitoring changes to the display, the method comprising: 
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executing a computer program at a server, wherein the execution of the computer program 
generates display objects from a set of display commands, the display objects are at least a portion of 
a display for said computer program; 

identifying changes of said display which are responsive to at least one type of continuous 
user interaction command received from a remote client 

determining whether said changes warrant an update to an image based at least in part on one 
or more of available bandwidth, available computing power, or type of user connection; and 

upon determining said changes do warrant an update, processing said changes and converting 
said display commands into a compressed video stream, wherein said changes are inserted into said 
compressed video stream at an update frame rate corresponding to a priority assigned to other 
portions of the display that are unchanged and such that changes to said image are inserted into the 
compressed video stream at a faster rate than compressed data that does not include changes to said 
image. 

129. A method according to claim 128, wherein said type of user command comprises a pointing 
device command. 

130. A method according to claim 128, wherein said changes comprises an indication of a 
selection of a GUI (graphical user interface) element. 

131. A method according to claim 128, comprising analyzing said user command to determine 
display commands which effect said identified changes. 

132. A method as recited in claim 94, wherein the compressed video stream is generated without 
first generating a display raster of at least the unmodified object. 
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133. A method as recited in claim 128, wherein the display commands are directly converted into 
the compressed video stream without first generating a display raster of the display commands. 

134. The method of claim 94, wherein the modification is performed in such a way that the 
modified object will be converted to compressed video faster than the unmodified object. 

135. The method of claim 94, wherein the modification includes replacing an object of the 
display, and wherein the replacement is based on the type of the object. 

136. The method of claim 94, wherein generating display object comprises generating display 
commands, wherein modifying comprises changing at least one of the display commands, and 
wherein the compressed video includes the at least one of the display commands after it is modified 
without first being rendered at the server. 

137. The method of claim 94, wherein the unmodified object is modified to produce the modified 
object by one or more of: adjusting a cursor flashing rate so that a lower frame rate is required; 
adjusting the colors of the unmodified object so that the compression of a color component is more 
efficient; or moving the unmodified object so as to not straddle compression-block boundaries. 

138. The method of claim 94, wherein the unmodified object is a display background, which is 
modified to produce the modified object by replacing the display background to simphfy 
compression or to utilize a pre-compressed background. 

139. The method of claim 128, wherein only changed portions of the image are processed and a 
new output frame is assembled that includes both newly compressed image portions and unchanged 
compressed image portions. 
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9. EVIDENCE APPENDIX 



Appendix Listing 
None 
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RELATED PROCEEDINGS APPENDIX 



Appendix Listing 
None 
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